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Figure  1.  One  of  our  CyberSeat  virtual  reality  setups  used  to  train  participants  to  navigate  through  a 
building,  in  this  case,  the  virtual  building  (Inset)  has  been  made  transparent. 
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^^patial  navigation  is  important  not  only  in  our 
idaily  routine,  but  in  a  wide  variety  of  com- 
^plex  critical  tasks.  For  example,  military 
forces,  law  enforcement  personnel,  and  fire-fighters 
all  encounter  situations  in  which  they  must  navigate 
through  unfamiliar  buildings  to  free  hostages,  dis¬ 
arm  defenses,  or  locate  people  in  need  of  rescue.  In 
these  circumstances,  exploring  the  actual  buildings 
is  impractical  or  impossible,  so  another  approach  to 
spatial  learning  is  needed  that  will  support  transfer 
to  performance  on  the  actual  tasks.  Virtual  reality 
technology  promises  one  potential  solution  to  this 
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need,  and  our  research  is  designed  to 
evaluate  that  approach.  We  are  interested 
both  in  how  well  virtual  environments 
(VE)  mimic  the  real  world  and  in  how  the 
VE  can  be  transformed  to  improve  on 
real-world  training. 

In  navigating  a  space,  it  is  generally 
acknowledged  that  there  are  two  pri¬ 
mary  types  of  knowledge^route  and 
survey.  Specific  paths  constitute  a 
“route”  and  enable  a  person  to  navigate 
from  one  location  to  another.  Such 
routes  are  fairly  specific  and  can  be 
thought  of  as  particular  procedures. 
When  a  more  global,  integrated  repre¬ 
sentation  is  developed,  the  person  is 
said  to  possess  “survey”  knowledge  of 
the  space.  Such  knowledge  enables  a 
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person  to  navigate  using  a  number  of  dif¬ 
ferent  paths.  Although  there  is  no  con¬ 
sensus  on  the  precise  nature  of  the 
underlying  representations,  it  is  generally 
recognized  that  these  two  types  of  knowl¬ 
edge  provide  for  rather  different  naviga¬ 
tional  skills. 

The  Cognition  and  Virtual  Reality 
Laboratory  at  The  Catholic  University  of 
America  has  focused  on  examining  the 
role  of  VE  in  developing  these  two  types 
of  knowledge.  This  paper  addresses 
three  relevant  questions  within  that 
framework — 

LDo  VEs  provide  an  effective  way  to 
learn  a  route? 

2.  Can  VE  improve  the  acquisition  of 
survey  knowledge  for  wayfinding 
and  object  localization? 

3.  Can  VE  change  the  character  or 
speed  of  survey  learning? 


Figure  2.  An  exterior  view  of  the  virtual  model  of  the  Crough  building  of  the 
School  of  Architecture  and  Planning  used  in  our  transfer  studies.  The  Inset 
shows  the  actual  building. 


Of  course,  answers  to  these  questions 
will  depend  both  on  what  is  presented  to  the  user 
and  on  the  interface  tools  controlling  that  presen¬ 
tation.  Our  emphasis  has  been  on  VE  that  can  be 
set  up  in  a  space  comparable  to  a  normal  work 


Figure  3,  An  interior  scene  of  the  virtual  Crough  building  used  in  our  exploratory  VE  studies.  The 
sphere  serves  to  enhance  exploration  of  the  building.  It  turns  from  red  to  green  once  that  space  has 
been  explored. 
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area  using  mid-range,  commercially  available 
hardware  and  software  (see  Figure  1,  page  1,  for  a 
typical  setup). 

Do  VEs  Provide  an  Effective  Way  to 
Learn  a  Route? 

In  a  series  of  studies,  we  examined  how  walk¬ 
throughs  in  a  VE  transferred  to  following  the  same 
route  in  the  actual  building.  Walkthroughs  along  a 
defined  route  in  VE  (see  Figures  2  and  3}  were  com¬ 
pared  with  tracing  the  same  path  on  a  floor  plan  or 
walking  the  route  in  the  actual  building.  When 
asked  to  follow  the  learned  route  in  the  real  build¬ 
ing  on  a  transfer  task,  those  trained  in  VE  per¬ 
formed  comparably  to  those  trained  in  the  actual 
building.  However,  when  the  transfer  route  in  the 
real  building  was  reversed,  the  VE  trained  group 
had  substantially  more  difficulty  than  either  of  the 
other  two  conditions.  This  pattern  of  results  was 
similar  when  people  were  tested  immediately  after 
learning  or  after  a  two-week  delay.  So,  for  purposes 
of  route-following,  VE  can  be  a  very  good  training 
method.  However,  VE  also  shows  substantially 
more  specificity  than  other  traditional  training 
methods— reversing  a  route  is  more  difficult  after 
VE  training  than  after  map  or  real-world  training. 
One  explanation  for  this  specificity  may  be  the 
restricted  field  of  view.  We  tested  this  hypothesis  by 
having  participants  learn  in  the  real  building  using 
goggles  that  limited  field  of  view  to  that  used  in  our 
head-mounted  display.  The  results  suggested  that 
the  field  of  view  only  accounts  for  a  portion  of  VE 
route  learning  specificity. 

Can  VE  improve  the  Acquisition  of 
Survey  Knowledge  for  Wayfinding  and 
Object  Localization? 

Although  learning  to  follow  a  specific  route  in  a 
VE  led  to  successful  transfer,  it  did  not  generalize 
well  to  other  routes.  Is  this  specificity  of  learning  a 
property  of  VE,  or  is  it  a  function  of  a  route  learning 
task?  To  answer  this  question,  we  examined  a  dif¬ 
ferent  form  of  VE  learning  in  which  people  explore 
the  same  large  building  space  on  their  own,  thus 
increasing  their  exposure  to  a  variety  of  viewpoints. 

When  instructed  to  “explore”  the  virtual  building 
until  they  were  familiar  with  its  spatial  layout,  peo¬ 
ple  learned  only  parts  of  the  space,  apparently 
unaware  of  the  limits  of  their  exploration.  We  there¬ 
fore  enhanced  the  virtual  space  with  a  series  of  red 
spheres  located  at  critical  points  in  the  building,  and 
instructed  people  to  locate  all  of  the  spheres.  When 
they  encountered  a  sphere,  audio  feedback  indicat¬ 
ed  the  current  location,  and  the  sphere  turned  green 
to  indicate  that  the  location  had  been  visited  (see 
Figure  3).  This  enhanced  cueing  improved  explo¬ 
ration  by  providing  a  mechanism  for  exploration 
feedback  not  available  in  a  physical  space. 


Transfer  effectiveness  was  assessed 
using  a  wayfinding  task  that  required 
people  to  go  from  one  location  to 
another  using  the  most  efficient  path. 
On  this  task,  training  using  a  VE  was 
far  more  efficient  than  training  using  a 
matched  floor  plan  of  the  building.  On 
average,  VE  paths  were  about  18% 
longer  than  optimal,  whereas  map 
paths  were  68%  longer. 

These  results  are  among  the  first 
demonstrations  that  virtual  training  can 
result  in  better  wayfinding  than  map- 
based  training.  Presumably,  exploratory 


Figure  4.  A  screen-shot  of  the  virtual  Community  Center  as  seen  in  the  transpar¬ 
ent  condition  which  enabled  people  to  see  through  floors  and  wails. 


VE  learning  provides  greater  exposure  to 
viewpoints  that  enhance  the  degree  of 
survey  knowledge. 

In  these  same  studies  we  examined 
how  well  people  would  do  in  identifying 
the  location  of  objects  they  had  found  in 
the  building.  Would  the  VE  serve  as  a 
better  tool  for  precise  object  location?  In 
a  cued-recall  task,  people  were  shown 
photographs  of  objects  they  had  seen  in 
the  building.  They  were  asked  to  identi¬ 
fy  the  objects*  location  in  the  actual 
building  using  either  a  map  or  a  VE  of 
the  building.  Those  people  who  had 
learned  the  space  in  a  VE  and  identified 
object  location  in  the  VE  performed 
markedly  better  (57%  of  object  placed 
within  5  feet  of  the  correct  location)  than 
those  who  had  either  learned  spatial  lay- 
continued  on  page  4 
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out  or  recalled  object  location  using  a 
map  (28%  within  5  feet). 

Presumably,  the  detailed  aspects  of  the 
display  improved  the  precision  of  object 
placement.  The  same  specificity  that  is  a 
detriment  in  route  following  may  be  an 
asset  in  learning  and  retaining  specific 
information  about  object  location. 

Can  Altered  VE  Change  the 
Character  or  Speed  of  Survey 
Learning? 

Although  most  studies  of  VE  training 
emphasize  the  mimicry  of  reality,  VR 
technology  also  makes  possible  changes 
from  reality  that  are  not  otherwise  possi¬ 
ble  in  real-world  training.  One  of  the 
problems  posed  by  learning  a  specific 
route  is  that  people  often  have  difficulty 
with  spatial  relationships  that  are  not 
along  that  route— they  lack  survey 
knowledge.  What  if  people  could  see 
relationships  that  were  not  constrained 
by  the  route  sequence  by  making  struc¬ 
tures  transparent,  the  “X-ray  vision” 
effect?  In  our  laboratory  we  evaluated 
this  approach  by  comparing  VE  training 
using  a  transparent-walled  building  (see 
Figure  4,  page  3)  with  that  using  a  typi¬ 
cal  opaque-walled  building.  We  then 
assessed  survey  knowledge  using  both 
drawings  and  a  speeded  response-time 
task  indicating  relative  location  of 
objects  in  the  building. 

VR  training  in  the  transparent-walled 
building  led  to  substantially  more  accu¬ 
rate  floor-plan  drawings  than  VR  train¬ 
ing  in  the  traditional  opaque-walled 
building.  This  advantage  was  mainly 
due  to  a  more  adequate  understanding 
of  the  architectural  aspects  of  the  build¬ 
ing's  layout  (e.g.,  connection  of  rooms 
and  alignment  of  floors) .  Mental  models 
developed  from  VR  training  reflect  prop¬ 
erties  of  the  simulated  environment. 
Learning  a  route  in  an  opaque  building 
led  to  a  mental  model  that  maintains 
route  distance — identifying  the  relative 
location  of  objects  in  such  a  building 
depends  upon  the  distance  along  the  fol¬ 
lowed  route,  or  what  is  known  as  a  city- 
block  metric.  In  contrast,  learning  a 
route  in  a  transparent  building  led  to  a 
mental  model  with  both  route  distance, 
reflecting  the  path  that  was  followed 
during  learning,  and  Euclidean  distance 
or  the  straight  line  distance  between 
objects  that  could  be  seen  through 


walls.  These  results  suggest  that  survey  knowledge, 
which  typically  requires  substantial  time  to  learn  in 
the  physical  world,  may  be  acquired  more  quickly 
using  VE  models.  In  addition,  transparency  may 
provide  a  useful  technique  in  the  development  of 
situation  awareness. 

Practical  Implications 

These  results  suggest  a  number  of  positive  ben¬ 
efits  for  VE  training  as  well  as  some  limitations. 
VE  can  be  useful  for  planning  or  mission  rehears¬ 
al  that  involve  spatial  navigation.  VE  route-fol¬ 
lowing  results  in  successful  transfer  to  the  same 
task  in  a  comparable  physical  space,  and  this  can 
be  extremely  useful  in  preparing  to  move  effi¬ 
ciently  to  a  given  location.  At  the  same  time,  this 
type  of  virtual  route  following  resulted  in  sub¬ 
stantial  specificity,  limiting  its  use  for  selecting 
alternative  routes.  Allowing  exploration  in  the  VE 
induces  more  flexibility  of  learning.  Building  cues 
into  the  VE  ensures  that  the  potential  space  of 
interest  is  actually  explored. 

The  relatively  detailed  visual  layout  of  a  VE  may 
also  provide  improved  reconnaissance.  When  a 
space  was  learned  using  VE  and  object  location 
was  recalled  using  that  VE,  accuracy  of  object 
placement  improved  dramatically.  It  is  an  open 
question  whether  this  recall  will  generalize  from 
specific  scenes  in  the  building  to  general  descrip¬ 
tions  of  objects. 

Finally,  altering  virtual  environments  offers  spe¬ 
cial  opportunities  for  modifying  learning.  Using 
transparent  environments  led  to  the  acquisition  of 
survey  knowledge  in  a  fraction  of  the  time  typi¬ 
cally  reported  for  real-world  training.  We  have 
only  begun  to  tap  the  links  of  virtual  and  real  spa¬ 
tial  navigation.® 

Author's  Note:  The  Cognition  and  Virtual  Reality 
Lnboratoiy  research  team  includes  Deborah  M. 
ClawsoiL  Ph.D.,  Benjamin  A.  Knott,  Ph.D.  (now  at 
Booz' Allen  &  Hamilton),  Michael  Miller,  M.A., 
Laura  Mnllin,  B.A.,  and  Michael  Piller,  B.S.  The 
work  reported  here  was  supported  by  the  Army 
Research  Institute,  contract  DASW01-96~000d  and 
by  the  Office  of  Naval  Research,  contract 
N00014-97-l~03SS.  The  content  of  this  article  does 
not  necessarily  reflect  the  position  or  policy  of  these 
organizations  or  of  the  ILS.  Government. 
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Virtual  Reality: 

A  Special  Issue  of  Gateway 


Benjamin  Knott 


Editor's  Note:  Benjamin  A.  Knott, 
Ph.IX,  an  Associate  at  Booz^  Allen  & 
Hamilton,  is  sewing  as  Guest  Technical 
Editor  for  this  issue  of  Gateway. 

his  issue  of  Gateway  is  the  first 
in  a  series  of  special  issues,  each 
of  which  will  focus  on  a  single 
theme.  The  objective  is  to  provide  a 
brief  overview  of  efforts  within  a 
field  that  is  of  interest  to  the  human 
factors  community.  This  issue  is 
dedicated  to  the  burgeoning  field  of 
Virtual  Reality  (VR) .  VR  is  a  method  of 
human-computer  interaction,  wherein 
a  number  of  advanced  interface  tech¬ 
nologies  are  employed  to  present  the 
user  with  the  illusion  of  being 
immersed  in  a  computer-generated,  or 
virtual,  environment.  This  method  has 
wide  implications  for  training,  medi¬ 
cine,  data  visualization,  design,  educa¬ 
tion,  and  entertainment,  or  any  area  in 
which  having  the  capability  to  simulate 
a  real  environment  can  be  of  benefit. 

While  the  idea  of  simulation  is  not 
new,  the  notion  of  “immersion”  in  a 
computer-generated  space  has  gained 
considerable  momentum  over  the  last 
decade,  spurred  on  by  the  emergence  of 
a  host  of  technological  innovations.  As 
the  term  “immersion”  suggests,  these 
innovations  support  a  close  coupling  of 
human  and  machine.  The  users  do  not 
simply  issue  a  sequence  of  commands  to 
a  computer,  but  rather  they  feel  that  they 
are  a  part  of  the  virtual  world.  However, 
to  fully  realize  the  potential  of  this 
approach,  a  great  deal  of  human  factors 
research  is  needed  to  determine  the  most 
effective  and  safe  manner  in  which  to 
implement  VR. 

There  are  so  many  varieties  of  VR  sys¬ 
tems  that  it  is  difficult  to  define  what 
exactly  constitutes  VR.  These  systems 
vary  widely  in  their  sophistication,  from 
fully  enclosed  flight  simulators  that 


include  detailed  models  of  specific  aircraft,  to  desk¬ 
top  implementations,  sometimes  called  “Fishtank 
VR,”  where  users  peer  into  a  virtual  world  through 
a  standard  computer  monitor.  While  there  is  no  pro¬ 
totypical  VR  system,  most  VR  systems  share  three 
basic  components— a  computing  environment,  and 
immersive  input  and  output  devices.  The  comput¬ 
ing  environment  consists  of  a  virtual  world  data- 
base,  software  for  rendering  that  database  dynami¬ 
cally  and  in  3-D,  and  software  for  managing  input 
from  the  user. 

Many  VR  systems  use  multiple  output  devices  to 
stimulate  several  of  the  users’  sensory  modalities. 
Visual  displays  usually  take  the  form  of  a  head- 
mounted  display  (HMD)  or  large  image  projection 
screens.  Stereo  images  can  be  used  to  enhance  the 
feeling  of  presence  in  the  virtual  environment.  In 
addition,  spatial  audio,  acoustic  signals  that  sound 
like  they  are  coming  from  a  specific  location  in  the 
virtual  world,  can  be  presented  through  head¬ 
phones  or  speakers.  Tactile,  force  feedback,  and 
inertial  displays  stimulate  a  person’s  sense  of  touch 
and  motion.  While  they  are  less  common,  they  can 
be  an  important  part  of  some  VR  applications. 
Examples  of  these  devices  include  gloves  that  apply 
pressure  or  vibration  to  the  skin  when  a  virtual 
object  is  touched,  or  motion  platforms  that  tilt  and 
roll  to  give  the  user  a  feeling  of  flight. 

Input  devices  usually  include  some  way  to  control 
navigation  and  position  tracking.  Methods  of  navi¬ 
gation  control  vary  considerably.  The  most  common 
is  a  joystick  or  a  mouse  that  allows  movement  in 
three  dimensions.  Others  include  treadmills  or  sim¬ 
ilar  devices  that  allow  one  to  use  more  natural  body 
movements  to  move  about  (e.g.,  see  article  entitled 
“Virtual  Environment  Technology  for  TYaining,” 
page  8,  in  this  issue  of  Gateway).  Position  tracking 
devices  sense  the  user’s  location  and  orientation 
with  respect  to  the  virtual  world  to  update  the 
appropriate  displays  (visual,  auditory,  or  tactile). 
They  are  typically  placed  on  the  HMD  to  track  the 
orientation  and  position  of  a  user’s  head,  allowing 
one  to  view  the  environment  from  any  number  of 
perspectives.  TYacking  of  the  hand  will  enable  a  per¬ 
son  to  grasp  and  move  objects  in  the  virtual  world. 
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The  technologies  mentioned  here  are  only  a  few 
of  the  possibilities.  There  are  a  variety  of  different 
types  of  HMDs,  gloves,  and  tracking  devices,  each 
with  unique  properties  and  performance  character¬ 
istics.  In  addition,  new  devices  continue  to  be  devel¬ 
oped  throughout  a  number  of  research  and  devel¬ 
opment  laboratories.  In  fact,  the  sheer  variety  of 
devices  and  the  ways  in  which  they  can  be  config¬ 
ured  pose  a  substantial  challenge  to  human  factors 
professionals.  To  date,  there  are  few  standards  or 
guides  for  VR  design,  and  few  sources  from  which 
to  gather  information  on  all  the  relevant  techniques. 
When  designing  a  virtual  environment,  for  a  train¬ 
ing  application,  the  designer  must  know  about  the 
alternative  technologies  that  exist,  how  they  are 
likely  to  affect  learning  and  transfer,  and  how  to 
produce  the  needed  effect  within  a  given  budget. 
There  is  a  growing  literature  on  the  human  factors 
of  VR,  but  the  problem  of  gathering  and  synthesiz¬ 
ing  the  research  into  useful  guidelines  for  develop¬ 
ment  is  considerable.  Perhaps  what  is  needed  is  a 
single  resource  for  information  regarding  VR 
research,  tools,  and  techniques. 

What  follows  is  a  series  of  articles  that  will 
explore  some  of  the  current  research  issues  and 
describe  promising  user-centered  applications  of 
VR.  The  success  of  VR  applications  of  the  future, 
whether  they  are  for  training  or  entertainment, 
depends  on  careful  consideration  and  research  into 
the  impact  of  the  various  technologies  on  human 
perception  and  learning.  This  issue  offers  a  glimpse 
into  that  future.* 


Why  We  Changed  Our  Name  to 


Human  Systems 


,  n  May  4,  2000  we  officially  changed  our  name  from  The  Human  Systems  lAC  mission  is  now  chartered  by 

(  I  “Crew  System  Ergonomics/Human  Systems  Director  of  Defense  Research  and  Engineering  (DDR&E)  to 

Technology  Information  Analysis  Center  collect,  analyze,  synthesize,  generate,  and  disseminate  sci- 

(CSERIAC)”  to  the  “Human  Systems  Information  Analysis  entific  and  technical  data  regarding  the  human  in  sea,  land. 
Center  (Human  Systems  lAC).”  air,  and  space  environments. 

The  reason  for  the  change  is  simple.  The  Human  Systems  Technical  areas  of  interest  include,  but  are  not  limited  to 
lAC  deals  with  the  human  component  of  a  system  using  a  general  human  factors  engineering,  ergonomics,  MPT  (man- 
“Total  System  Approach”  in  accordance  with  Department  of  power,  personnel,  and  training),  personnel  survivability  fac- 
Defense  (DoD)  policy  to  stress  the  importance  of  optimizing  tors,  health  hazards,  safety  factors,  medical  factors,  human 
total  system  performance  and  minimizing  the  hfe-cycle  cost  characteristics,  performance-related  factors,  human  comput- 
of  ownership.  The  total  system  includes  not  just  the  prime  er  interactions,  design  of  workstations  and  facilities,  system 
mission  equipment,  but  also  the  people  who  operate,  main-  characteristics,  work  design,  standards,  guidelines,  codes  of 
tain,  and  support  the  system;  the  training  and  training  practice,  and  organization,  social,  economic  and  political 
devices;  and  the  operational  and  support  infrastructure.  aspects  of  ergonomics,  methodology  for  research,  test  and 

evaluation,  and  other  corresponding  initiatives. 


The  Observer’:  as  flexible 
as  human  behavior 


The  professional  software  system  for  observing,  coding  and  analyzing  series  of  events:  activities,  postures, 
movements,  positions,  facial  expressions,  social  interactions  or  any  other  aspect  of  human  behavior, 

•  The  Observer  Basic  -  Software  for  collection,  analysis,  management  and  presentation  of  observational 
data.  For  live  data  entry  by  a  human  observer,  using  a  desktop  or  handheld  computer. 

•  The  Observer  Mobile  •  Systems  for  mobile  collection  and  analysis  of  observational  data. 

Each  kit  contains  The  Observer  software,  a  handheld  computer  and  accessories. 

^  •  The  Observer  Video-Pro  -  Professional  system  for  coding,  annotating, 

editing  and  analyzing  behavioral  processes  from  video.  Combines  The  Observer  soft- 
ware  with  the  latest  developments  in  video  time  code  and  multimedia  technology. 
UtinfM  Supports  analog  video  tape,  digital  video  tape  and  digital  video  on  disk. 


Human  factors  and  ergonomics  applications: 

•  Analyzing  human-machine  •  Safety  and  risk  assessment 

interaction  •  Analyzing  driving  behavior 

'Virtual  reality  simulations 
'Analyzing  group  processes 
'  Working  posture  analysis 

•  Usability  testing  '  Time  and  motion  studies 

'  Analyzing  decision  making  '  and  much  morel 
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Virtual  Environment  Technology  for  Training 


Nika  IVlatzke  and 
Dylan  Schmorrow 


'Virtual  Environments  (VEs)  pres¬ 
ent  realistic  simulations  of  inter¬ 
active  scenes.  The  Office  of 
Naval  Research  (ONR)  has  ongoing 
efforts  aimed  at  developing  VE  systems 
to  provide  readily  available,  low-cost, 
and  portable  devices  to  train  personnel 
on  a  broad  range  of  operational  and 
other  skills.  The  training  activities  will 
range  from  maintenance  operations  to 
situation  awareness  and  decision  strate¬ 
gies  in  a  battlefield  setting.  VEs  will  pro¬ 
vide  opportunities  to  train  in  situations 
that  are  either  too  costly  or  impractical 
to  execute  with  traditional  types  of 
training  exercises.  VEs  will  also  enable 
trainers  to  measure  perform 
ance  more  accu¬ 
rately  and  effi 
ciently  than 
can  be  done 
through  tradi¬ 
tional  training 
regimens.  Having 
such  performance  meas 
ures  and  additional  training 
opportunities  before  going  into  live  mis¬ 
sions  will  have  a  significant  impact  in 
timesavings,  accident  prevention,  and 
survivability.  Among  the  products  of 
the  ONR  VE  program  to  date  are  VEs  to 
train  operators  in  submarine  piloting, 
shiphandling,  and  remotely  operated 
vehicle  piloting. 

VEs  have  great  potential  in  military 
training  applications,  yet  the  current 
capabilities  are  limited.  VE  training  sys¬ 
tems  are  computationally  challenging. 
They  require  complex  modeling  of 
human  perception  and  cognition,  and 
they  are  currently  limited  to  a  very 
restricted  range  of  task  domains.  Future 
systems  need  more  flexibility  and  should 
be  generic  enough  that  they  can  be 
reconfigured  for  application  in  a  multi¬ 
tude  of  contexts.  Another  potential  direc- 


tion  is  networking  multiple  VE  systems  over  distrib¬ 
uted  sites  for  interactive  VE  use.  Distributed  net¬ 
works  of  VE  systems  will  allow  users  to  interact 
with  one  another,  creating  a  convenient  and  power¬ 
ful  mode  of  relaying  information  and  training  per¬ 
sonnel.  There  are  several  areas  that  the  ONR  VE 
program  will  investigate  to  improve  current  capabil¬ 
ities  of  VEs.  One  that  is  particularly  relevant  to 
human  factors  is  perception. 

Perceptual  Issues  in  VEs 

A  very  important  and  often  overlooked  aspect  of 
the  development  and  design  of  emerging  technolo¬ 
gies  is  human  sensation  and  perception.  The  ubiq¬ 
uitous  and  loosely  used  term,  “multimodal,”  refers 
to  the  use  of  multiple  sensory  input  in  VE  systems. 

Many  systems  that  are  deemed  multi¬ 
modal  are  at  best  bimodal 
(e.g.,  incorporating  visu¬ 
al  and  haptic  input) . 
Fully  immersive  multi¬ 
modal  systems  will  con¬ 
tain  several  sensory-per¬ 
ceptual  modes  such  as  visu- 
auditory,  haptic,  kinesthetic,  and 
proprioceptive  input.  Understanding  how 
our  sensory  perception  operates  will,  among  other 
things,  improve  depth  perception  in  VEs,  facilitate 
production  of  apparent  motion,  and  increase  the 
accuracy  of  spatial  relationships  and  distance  judg¬ 
ments  in  VEs. 

Future  Directions 

The  ONR  VE  program  has  access  to  an  excellent 
group  of  researchers  who  have  helped  us  shape  the 
future  of  the  program.  Below  are  some  of  the  excit¬ 
ing  new  projects  along  with  some  ongoing  projects 
that  will  be  pursued  in  the  years  ahead. 

Sensory  Perceptual  Issues  Beyond 
High-Fiflelity  Graphics 

Spatial  audio,  virtual  sound,  and  the  integration 
of  virtual  with  real  sound  will  be  among  the  future 
components  of  VEs.  Although  sound  likely  adds  to 
the  fidelity  of  a  VE,  for  the  purposes  of  the  ONR  VE 
program,  the  value  of  auditory  input  in  VEs  will 
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Figure  1.  LCDR  Dylan  Schmorrow  in  an  immersive 
computer-generated  3-D  environment  developed  by 
the  Institute  for  Simulation  and  Training,  University  of 
Central  Florida. 

need  to  be  examined.  For  example,  researchers  will 
need  to  examine  whether  sound  in  VEs  enhances 
situation  awareness,  increases  the  accuracy  of  local¬ 
izing  objects,  or  aids  in  judging  location  relative  to 
sound  sources  (i.e.,  to  aid  in  navigation). 

Initial  studies  investigating  the  use  of  sound  in 
VEs  for  distance  and  location  estimates  of  objects 
showed  that  localization  of  sound  is  very  difficult 
unless  the  sound  is  stationary.  In  fact,  humans  are 
much  more  accurate  using  vision  as  opposed  to 
sound  for  distance  estimation.  That  finding  may 
partially  be  due  to  limitations  of  human  sensory 
capacity,  but  also  due  to  current  limitations  of  avail¬ 
able  sound  devices.  As  technological  advances  and 
improvements  in  spatial  audio  occur,  the  value  of 
auditory  input  in  VEs  is  more  likely  to  be  seen. 

“Haptic”  refers  to  the  sense  of  touch  and  force 
feedback  information  from  the  muscles  and  joints. 
Haptic  devices  will  be  used  for  various  purposes 
including  use  as  a  navigational  aid.  For  instance, 
pilots  could  receive  haptic  input  for  signaling  their 
direction  or  location.  Another  potential  application 
of  haptic  devices  will  be  the  manipulation  and  inter¬ 
action  with  objects  in  VEs. 

Including  inputs  that  provide  a  sense  of  aware¬ 
ness  of  movements  and  position  of  the  body  (pro¬ 
prioception)  that  is  independent  of  vision  in  VEs 
also  enhances  performance  on  certain  tasks.  In 
one  study,  researchers  showed  that  self-propelled 
locomotion  increases  accuracy  in  localization  of 
objects  and  judgments  of  position  relative  to  sta¬ 
tionary  objects. 

While  self-locomotion  is  clearly  a  value-added 
navigational  aid,  not  all  VEs  will  have  the  space  for 
the  user  to  move  about  unconstrained.  Thus  meth¬ 
ods  to  mimic  kinesthesia  (the  illusion  of  moving  in 
space)  are  likely  to  be  further  developed.  For  exam¬ 
ple,  the  Naval  Research  Laboratory  is  developing  a 
locomotion  interface  called  “Gaiter”  that  permits  a 


natural  walking-in-place  interface  for 
moving  about  in  large-scale  VEs.  That 
particular  system  uses  wireless  tracking 
devices  to  monitor  the  user’s  movements 
walking  or  running  in  place,  and  moves 
the  user’s  viewpoint  through  VE  to  give 
the  user  the  impression  of  virtual  move¬ 
ment.  Even  though  the  user  knows  he  is 
walking  in  place,  presenting  optical  flow 
scaled  to  correlate  with  the  user’s  knee 
motion  makes  it  possible  for  a  user  to 
suspend  his  disbelief  and  participate  in 
the  illusion  of  false  motion. 

Of  course,  a  drawback  to  incorporat¬ 
ing  perceptual  input  in  VEs  is  the 
potential  adverse  side  effects  from  per¬ 
ceived  and  apparent  motion.  Current 
systems,  including  those  that  utilize 
head-mounted  displays  (HMD)  (see 
Figure  1),  may  have  negative  effects  on 
the  user  such  as  motion  sickness  and 
postural  instability.  Studies  with  HMDs 
have  shown  that  factors  contributing  to 
adverse  side  effects  include  visual 
update  delays,  field  of  view,  and  reso¬ 
lution  quality.  Additional  side  effects 
can  occur  when  VE  systems  are  used 
on  moving  platforms  such  as  ships. 
Therefore,  efforts  to  circumvent  or  mit¬ 
igate  adverse  effects  are  an  important 
part  of  the  ONR  VE  program. 

Stmuiating  Natural  Actions  for 
Close-quarters  Battle 

Simulating  real-life  environments 
requires  providing  input  to  and  tracking 
output  from  the  user.  The  input  can 
include  providing  the  user  with  a  sense 
of  a  representation  of  his  body  (avatar). 
Having  the  user  make  contact  with  vir¬ 
tual  objects  can  also  help  simulate  natu¬ 
ral  interaction.  The  application  platforms 
for  future  VE  systems  range  from  sub¬ 
mersible  vehicles  to  computer-generated 
ground  forces  (see  example  in  Figure  2). 

i\^otlifiable  VEs  for  Distributed 

interaction 

Basic  and  reconfigurable  scenario 
development  will  be  another  future 
direction  for  VE  systems.  VEs  should 
have  the  capability  to  be  modified,  as  the 
users’  training  needs  change. 
Additionally,  the  systems  will  eventually 
be  networked  so  that  instructors  can 
reach  a  large  number  of  trainees  at  dis¬ 
tributed  sites  and  so  that  the  trainees  can 
interact  with  one  another.  The  major 
continued  on  next  page 
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continued  from  previous  page 

For  more  information, 
please  contact: 

Dylan  D.  Schmorrow,  Ph.D. 
LCDR,  MSC,  USN 
Information  Technology 
Division 

Naval  Research  Laboratory 
4555  Overlook  Avenue,  SW 
Washington,  DC  20375 

Tel:  202-404-8624 

Fax:  202-404-7887 

E-mail:  schmorro@ait.nrl. 
navy.mil 

Sarah  Monique  (Nika) 
Matzke,  Ph.D.,  is  a  Program 
Specialist  in  the  TYaining  and 
Virtual  Environment  Program, 
Cognitive  Neural  Science  and 
Technology  Division,  Office  of 
Naval  Research,  Washington, 
DC,  and  Lieutenant  Commander 
Dylan  Schmorrow,  Ph.D.,  is  the 
Military  Assistant  for  Modeling 
and  Simulation  Science  and 
Technology  in  the  Information 
Technology  Division,  Naval 
Research  Laboratory, 

Washington,  DC,  and  as  the 
Human  Systems  Department 
Program  Officer,  Office  of  Naval 
Research,  Washington,  DC. 


Figure  2.  A  visionary  example  of  VE  systems  that  will  encompass  blue-water  simulations  to  com¬ 
puter-generated  forces  on  the  ground. 


appeal  of  a  distributed  approach  for  the 
military  is  the  ability  to  deploy  the  train¬ 
ing  device  along  with  military  personnel. 

Tactical  Navigation 

A  major  use  of  VEs  has  been  and  will 
continue  to  be  the  training  of  spatial 
skills  for  navigation.  Spatial  abilities 
involve  the  acquisition  and  use  of  spa¬ 
tial  knowledge  used  for  wayfinding, 
search,  map  use,  and  direction  and  dis¬ 
tance  estimation.  Haptic  auditory,  and 
kinesthetic  or  proprioceptive  types  of 
information  as  navigation  aids  are 
important  areas  to  investigate  for  their 
usefulness  in  tactical  navigation. 


Keeping  Up  With  the 
ONR  VE  Program 

VEs  are  exciting  new  tools  to  exploit  for  training 
and  other  purposes.  Their  success  in  training 
applications  will  likely  depend  upon  the  measura¬ 
ble  transfer  from  the  VE  to  real-world  execution. 
Positive  transfer  may  depend  upon  the  fidelity  of 
the  VE  itself.  How  “real”  the  VE  must  be  to  facili¬ 
tate  learning  and  retention  will  be  one  of  the 
many  questions  the  ONR  VE  program  intends  to 
answer.  E 


Instriimenlation  System  for  Analysis 
of  Human  Peilormance 

A  key  component  of  training  in  VEs 
will  be  the  ability  to  predict  performance 
in  the  real  world.  Methods  for  tracking 
and  analyzing  the  user's  actions  will 
likely  be  developed  and  included  in  VE 
systems.  While  training  itself  should  pro¬ 
duce  improvement  in  performance 
measures,  providing  the  user  with  feed¬ 
back  from  analyses  of  the  user’s  actions 
should  also  help  improve  performance. 
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Virginia  Beach,  VA,  USA.  June  20-22,  2000 

Threats,  Countermeasures,  and  Situational  Awareness: 

Teaming  for  Survivability  Symposium  and  Exhibition 

Contact  Mr.  Tom  Assenmacher,  Naval  Air  Warfare  Center  Aircraft  Division,  Human 
Engineering  Applications  Branch  (4. 6.4.2),  Building  2187,  Suite  2280,  48110  Shaw  Road, 
Unit  5,  Patuxent  River,  MD  20670-1906,  USA.  Tel:  +1-301-342-0026,  DSN:  342-0026, 
Fax:  +1-301-342-9305,  E-mail:  AssenmacheTJ@navair.navy.mil 


San  Diego,  CA,  USA.  July  30-August  4,  2000* 

iEA-2000/HFES-20Q0.  International  Ergonomics  Association  and  Human  Factors  and  Ergonomics 
Society  Annual  Meetings 

Contact  HFES,  PO  Box  1369,  Santa  Monica,  CA  90406-1369,  USA. 

Tel:  +1-310-394-1811,  Fax:  +1-310-394-2410,  E-mail:  hfes@compuserve.com, 

URLs:  http://iea2000.hfes.ora  and  http://hfes.org 


Ann  Arbor,  Ml,  USA.  August  7-18,  2000 

Human  Factors  Engineering  Short  Course 

Contact  Center  for  Professional  Development,  University  of  Michigan  College  of 
Engineering,  2121  Bonisteel  Boulevard,  Ann  Arbor,  Ml  48109-2092,  USA. 

Tel:  +1-734-647-7200,  Fax:  +1-734-647-7182, 

URL:  http://www.umich.edu/~drivina/shortcourse/index.html 


Crieff,  Scotland.  September  4^8, 2000 

The  24'"  Conference  of  the  European  Association  for  Aviation  Psychology  (EAAP) 
Contact  Dr.  Malcolm  James  Cook,  University  of  Abertay,  1 58  Marketgait,  Dundee, 
Scotland  DD1  1NJ.  Tel: +44-1382-308178,  Fax: +44-1382-223121 ; 

E-mail:  mcook@abertay.ac.uk,  URL:  http://www.eaap.com 


Lexington,  KY,  USA.  September  26-28,  2000 

SAE’s  Southern  Automotive  Manufacturing  Conference  and  Exposition 

Contact  Society  of  Automotive  Engineers,  400  Commonwealth  Drive,  Warrendale,  PA 

15096-0001,  USA.  Tel:  +1-724-776^841,  Fax:  +1-724-776-0210, 

URL:  http://www.sae.ora 


Toulouse,  France.  September.  27-29,  2000 

International  Conference  on  Human-Computer  Interaction  in  Aeronautics,  HCi-Aero  2000 
Contact  Ms.  Helen  Wilson,  HCI-Aero  2000  Office,  European  Institute  of  Cognitive 
Sciences  and  Engineering  (EURISCO),  4  Avenue  Edouard  Belin,  31400  Toulouse, 
France.  Tel:  +33-5-62-17-38-38,  Fax:  +33-5-62-17-38-39, 

E-mail:  wilson@onecert.fr,  URL:  http://www-eurisco.onecert.fr/ 
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XIVth  Triennial  Congress  of  the 
International  Ergonomics  Association 


44th  Annual  Meeting  ofthe 

Human  Factors  and  Ergonomics  Society 


General  Chair 
Hal  W.  Hendrick 
Technical  Program  Chair 
Ogden  Brown,  Jr. 

HFES  Technical  Program 
Committee  Cochairs 
Clint  A.  Bowers 
John  F.  (J^fD  Kelley 
Assistant  Technical 
Program  Chair 
Waldemar  Karwowski 
Sponsorships  Chair 
Kenneth  R.  Laughery,  Sr. 
lEA  Liaison 
Y.  Ian  Noy 

Latin  America  Liaison 
Stuart  O.  Parsons 
At-Large  Member 
Andrew  S.  Imada 
Tours 

Robert  J.  Smillie 
Volunteers 
Suzanne  M.  Dawes 
Hospitality 
Michael  J.  Welch 
Media  &  Public  Relations 
O.  Keith  Hansen 
On-Site  Information 
Sherry  K.  Proctor 


July  30-August  4,  2000 
San  Diego  Marriott  Hotel  and  Marina 
San  Diego,  California  USA 

Join  us  in  San  Diego  this  summer  for  the  largest  gathering  of  human  factors/ 
ergonomics  professionals  in  history.  The  Congress  features  hundreds  of  technical 
sessions  covering  these  areas  and  more: 

Aerospace  Systems 

Aging 

Agriculture 

Cognitive  Ergonomics 

Communications 

Computer  Systems 

Environmental  Design 

Industrial  Ergonomics 

Macroergonomics 

Medical  Systems  and  Rehabilitation 

Safety 

Standards 

Test  and  Evaluation 

Transportation 

Virtual  Environments 

Choose  from  18  hands-on  workshops  to  be  held  the  weekend  prior  to  the 
Congress.  Each  day's  technical  program  begins  with  invited  addresses  from  leading 
ergonomics  professionals  from  around  the  world.  Also  visit  the  exhibit  hall,  tour 
technical  facilities  in  the  San  Diego  area,  and  enjoy  special  social  events  and  sight¬ 
seeing  tours.  Registration  and  program  information  is  now  available. 

For  details,  visit  the  Congress  Web  site:  http://iea2000.hfes.org 

or  contact  the  Human  Factors  and  Ergonomics  Society 

RO.  Box  1369,  Santa  Monica,  CA  90406-1369  USA 
310/394-1811,  Fax  310/394-2410 
http://hfes.org 
hfes@compuserve.com 
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*Look  for  the  Human  Systems  lAC  exhibit  at  these  meetings! 


Reno,  NV,  USA.  October  9-11,  2000 

SAFE  Association  38'"  Annual  Symposium 

Contact  SAFE  Association,  107  Music  City  Circie,  Suite  112,  Nashville,  TN  37214,  USA. 
Tel:  +1-615-902-0056,  Fax:  +1-615-902-0077,  E-mail:  safe@usit.net, 

URL:  http://safeassociation.ora/contact.htm 


Dayton,  OH,  USA.  October  10-12,  2000 

National  Aerospace  and  Electronics  Conference,  NAECON  2000 

Contact  Dr.  D.  W.  Repperger,  Technical  Program  Chair,  NAECON  2000,  833  Blossom 
Heath  Road,  Dayton,  OH  45419-1102,  USA.  Tel:  +1-937-255-8765, 

E-mail:  d.repperger@ieee.org,  URL:  http://www.naecon.org 


Savannah,  GA,  USA.  October  15-19,  2000 

Human  Performance,  Situation  Awareness  and  Automation: 

User-Centered  Design  for  the  New  Millennium 

Contact  Dr.  David  Kaber,  Dept,  of  Industrial  Engineering,  PO  Box  9542,  Mississippi  State 
University,  MS  39762,  USA.  E-mail:  kaber@engr.msstate.edu.  Or  contact  Dr.  Mica 
Endsley,  SA Technologies,  Inc.,  4731  E  Forest  Peak,  Marietta,  GA  30066,  USA. 

E-mail:  mica@satechnologies.com,  URL:  http://www.ie.msstate.edu/hpsaa/index.html 


Edinburgh,  Scotland.  October  25-27,  2000 

Third  International  Conference  on  Engineering  Psychology  and  Cognitive  Ergonomics 
Contact  Dr.  Don  Harris,  Human  Factors  Group,  College  of  Aeronautics,  Cranfield 
University,  Cranfield,  Bedford  MK43  OAL,  UK.  Tel:  +44-1234-750111,  ext.  5196, 
Fax:  +44-1234-750192,  E-mail:  icep@cranfield.ac.uk, 

URL:  http://www.cranfield.ac.uk/coa/coa  conf.htm 


El  Paso,  TX,  USA.  November  6-9,  2000 

45*"  Biennial  Meeting  of  the  U.S.  Department  of  Defense  Human  Factors  Engineering  Technical 
Advisory  Group 

Contact  Ms.  Sheryl  Cosing,  10822  Crippen  Vale  Court,  Reston,  VA  20194,  USA. 

Tel:  +1-703-925-9791,  Fax:  +1-703-925-9644,  E-mail:  sherylynn@aol.com, 

URL:  http://dticam.dtic.mil/hftag/  Meeting  is  open  to  ait  government  personnei  and  others 
by  specific  invitation. 


nov 


Orlando,  FL,  USA.  November  27-30,  2000* 

interservice/industry  Training,  Simulation,  and  Education  Conference  (l/ITSEC  2000) 
Contact  Ms.  Barbara  McDaniel,  National  Training  Systems  Association,  One  Coionial 
Place,  2111  Wilson  Boulevard,  Suite  400,  Ariington,  VA  22201-3061,  USA. 

Tel:  +1-703-247-2569,  Fax:  +1-703-243-1659,  E-mail:  bmcdaniel@ndia.org, 

URL:  http://www.iitsec.org 
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Warfighter-centered 

Virtual  Reality  Technology 


iffi^he  Human  Effectiveness 
jg  Directorate  of  the  Air  Force 
B  Research  Laboratory  (AFRL)  initi¬ 
ated  its  research  and  exploratory  devel¬ 
opment  of  warfighter-centered  virtual 
reality  technology  in  the  1940s,  well 
before  the  term  "virtual  reality”  was 
coined.  Flight  simulation  provided  the 
original  need  for  virtual  reality  beginning 
with  the  “Link”  trainer  of  the  type  used 
during  World  War  II  for  teaching  pilots 
proper  procedures  of  instrument  flight. 
For  many  years,  flight  simulation  was  the 
major  impetus  for  virtual  reality 
advances  in  areas  such  as  visual  sys¬ 
tems,  cueing,  and  the  simulation  of  high¬ 
ly  realistic  aerodynamic  modeling.  As 
defined  in  1992  by  the  National  Research 
CoundFs  Committee  on  Human  Factors, 

“Virtual  Reality  is  the  experience  of  being  in  a  synthetic 
(simulated)  environment  and  of  perceiving  and  interact¬ 
ing  through  sensors  and  effectors,  actively  or  passively, 
with  it  and  the  objects  in  it  as  if  they  were  real.  ” 

The  Human  Effectiveness  Directorate 
continues  to  lead  technical  innovation  in 
the  virtual  reality  technologies  of  aircrew 
training,  flight  simulation,  and  partially 
immersive  warfighter  interfaces. 

Training  and  Flight  Simulation 

The  Human  Effectiveness  Directorate  of 
AFRL  (and  its  predecessors)  has  had  a 
major  part  to  play  in  both  advancing  vir¬ 
tual  reality  for  flight  simulation  technolo¬ 
gies  and  in  understanding  how  to  best  use 
those  technologies  to  train  warfighters. 
Current  virtual  reality  technology  areas 
that  the  Directorate  is  pursuing  to  improve 
flight  simulation  for  training  include  visu¬ 
al  systems,  representation  technology,  and 
interconnection  technology. 


iVlichael  W.  Haas 
Deo  H.  Andrews 


Visual  systems  technology  can  be  divided  into 
three  interactive  components.  The  database  con¬ 
tains  information  about  the  terrain  and  man-made 
objects  that  are  to  be  represented  in  the  visual 
scene.  The  image  generator  takes  the  information 
from  the  database  and  allows  its  visual  represen¬ 
tation  to  be  displayed  to  the  trainee.  The  visual 
display  itself  is  the  third  technology.  The 
Directorate  has  significantly  contributed  vital 
improvements  to  all  three  technologies  over  the 
years.  For  example,  the  current  display  of  choice 
throughout  the  world  for  portraying  out-the-wind- 
screen  visuals  is  a  virtual-real  image  display 
developed  within  the  Directorate  (see  Figure  1). 
Also,  the  Directorate  has  worked  closely  with 
industrial  partners  to  improve  the  database  gener¬ 
ation  and  image  generation  capability,  as  well  as 
new  forms  of  visual  display.  Due  to  this  work,  the 
long-held  goal  of  providing  eye-limiting  resolution 
in  simulator  visual  systems  is  within  reach. 


Figure  1 .  A  pilot  in  the  F-~16C  IVlulti-task  Trainer  flying 
in  a  high-resolution  database  in  a  virtuaLreal  image 
display.  The  entire  system  was  developed  at  HEAs 
Mesa,  Arizona  facility.  The  database  depicts  Nellis 
AFB,  Nevada.  The  resolution  of  the  virtual  image 
around  the  air  base  is  0.5  meters.  As  the  pilot  flies  to 
the  Nellis  Ranges,  the  resolution  drops  off  to  about 
4-5  meters. 
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To  provide  realistic  warfighter  training  it  is  vital 
that  the  virtual  battlespace  be  populated  with  real¬ 
istic  entities,  either  via  human-in-the-loop  simula¬ 
tions  or  through  computer-generated  synthetic 
forces.  The  Directorate  has  worked  closely  with  the 
computer-generated  forces  community  to  help  pro¬ 
vide  valid  and  accurate  models  of  warfigher  cogni¬ 
tion  and  behaviors.  Realistic  threat  systems  and  vir¬ 
tual  enemy  and  friendly  aircrews  allow  trainees  to 
hone  their  skills  against  intelligent  forces  that  repre¬ 
sent  the  actual  battlespace. 

The  Directorate  has  also  been  an  active  contribu¬ 
tor  to  the  improvement  and  application  of  intercon¬ 
nection  technology  and  protocols  for  linking  virtual 
(human-in-loop  simulators),  live  (aircraft  on 
ranges),  and  constructive  (computer-generated 
threats  and  forces)  training  assets.  This  intercon¬ 
nection  technology  has  now  made  it  possible  for 
warfighters  to  link  to  training  assets  literally  around 
the  world  so  they  can  enter  training  scenarios  with 
disparate  forces  on  an  on-demand  basis.  That 
means  an  aircrew  can  literally  sit  in  an  F-15C  sim¬ 
ulator  cockpit  at  Eglin  AFB,  Florida  and  be  con¬ 
nected  to,  for  example,  F-15C  human-in-the-loop 
simulators  at  Langley  AFB,  Virginia,  F-16C  simula¬ 
tors  at  Shaw  AFB,  South  Carolina,  and  AWACS  real 
aircraft  at  Tinker  AFB,  Oklahoma.  This  ability  to 
link  to  virtual,  live,  and  constructive  assets  provides 
a  powerful  synthetic  battlefield  that  can  be  accessed 
from  anywhere  in  the  world. 

Interface  Technology  for  the  Warfighter 

Beginning  in  the  1960s,  miniature  cathode  ray 
tube  components,  lightweight  head-mounted  optics, 
high-voltage  connectors,  and  position/attitude  hel¬ 
met-tracking  systems  were  the  initial  technologies 
developed  within  the  Human  Effectiveness 
Directorate  for  integration  into  helmet-mounted  dis¬ 
plays  and  sights.  Then,  as  well  as  now,  a  helmet  dis¬ 
play,  when  combined  with  an  accurate  head-track¬ 
ing  system,  enables  intuitive  control  of  complex 
weapon  systems  and  a  spatially  stabilized,  partially 
immersive  visual  environment.  As  these  technolo¬ 
gies  were  matured  and  integrated  into  helmet  sys¬ 
tem  prototypes  by  the  Human  Effectiveness 
Directorate,  performance  and  affordability  improved 
leading  to  flight  simulator  evaluations,  flight  testing, 
and  the  establishment  of  a  joint  service  Helmet- 
mounted  System  Acquisition  Program  Office.  It  is 
possible  that  in  future  aircraft,  the  stabilized  dis¬ 
plays  of  a  helmet  system  may  replace  the  heads-up 
display  common  to  many  current  combat  aircraft. 

Increased  weapon  system  performance  gained  by 
utilizing  a  partially  immersive  visual  environment 
appears  to  be  amplified  when  the  visual  environ¬ 
ment  is  supplemented  with  auditory  and  haptic  dis¬ 
plays.  The  Human  Effectiveness  Directorate  is  con¬ 
tinuing  its  pioneering  development  of  displays 


which  spatialize  multiple  audio  signals  in  For  more  information, 
real  time.  please  contact: 

The  Directorate  is  also  innovatively 
combining  head-mounted  visual  dis-  Michael  W.  Haas,  Ph.D. 
plays,  spatial-audio  displays,  and  haptic  AFRL/HECP 
displays  into  prototype  crew  stations  and  2255  H  Street 
evaluating  them  in  research  flight  simu-  AVPAFB,  OH  45433-7022 
lators  for  inhabited  and  uninhabited  air¬ 
craft.  This  research  and  development  Tel:  937-255-8768 

have  attracted  international  interest  and  DSN:  785-8768 
led  to  collaborations  with  countries  such  E-mail:  michael.haas® 
as  France,  the  United  Kingdom,  wpafb.af.miI 

Australia,  and  Sweden  (see  Figure  2). 

Most  recently,  researchers  within  the  Michael  W.  Haas,  Ph.D.,  is  a 
Human  Effectiveness  Directorate  have  Principal  Electronics  Enffneer 
established  a  paradigm  shift  in  the  with  the  Human  Interface 
design  of  interfaces  incorporating  partial-  Technology  Branch,  Crew 
ly  immersive  displays.  The  shift,  simply  Systems  Interface  Division, 
put,  is  that  the  system  with  which  an  Human  Effectiveness 

operator  interacts  needs  to  be  capable  of  Directorate,  U.S.  Air  Force 
estimating  the  mental  and  physical  state  Research  Laboratory,  Wright- 
of  the  operator  using  physiologic  and  PattersonAFB,OH,andDeeH. 
behavioral  indices,  and  based  upon  that  Andrews,  Ph.D.,  is  a  Technical 
estimate,  adapt  its  behavior  to  best  sup-  Advisor,  Warfighter  Training 
port  the  operator.  In  essence,  systems  of  Research  Division,  Human 
the  future  will  react  to  the  perceptual.  Effectiveness  Directorate,  U.S. 
cognitive,  and  physical  capabilities  of  the  At  Farce  Research  Laboratory, 
operator,  perl^aps  even  evolving  over  Mesa,  AZ. 
time.  This  novel  direction  in  design  is 
immature  but  holds  great  promise  for 
increasing  the  effectiveness  of  humans 
interacting  with  complex  systems  which 
incorporate  partially  immersive  displays 
operating  in  highly  dynamic  and  time- 
critical  military  environments.* 
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Simulator  Sickness  in 

Virtual  Environments 


Bob  Witmer  & 
Donald  Lampton 


Figure  1 .  VE  Interface  for  Team 
Training  Research  developed  by 
ARi  and  the  Institute  for 
Simulation  and  Training. 


I^he  U.S.  Army  has  invested  heav- 
i  ily  in  the  use  of  virtual  environ- 
3  ments  (VE)  for  training. 
Beginning  in  the  1980s  with  networked 
simulators,  the  Army  has  committed  to 
use  virtual  simulations  to  train  combat 
forces  and  to  evaluate  new  systems  and 
operational  concepts.  These  simulations 
have  focused  on  training  personnel  who 
fight  from  within  combat  vehicles. 

More  recently,  the  need  to  train  dis¬ 
mounted  soldiers  in  these  simulations 
has  been  recognized.  Responding 
to  this  need,  the  U.S.  Army 
Research  Institute  for  the 
Behavioral  and  Social 
Sciences  (ARI), 

Simulator  Systems 
Research  Unit,  initiated 
a  research  program  in 
1992  to  investigate  the 
use  of  VE  technology  to 
train  dismounted  soldiers. 

After  analyzing  dismounted 
soldier  tasks  and  reviewing  the 
VE  training  literature,  we  investigated 
interface  effects  on  users’  capability  to 
perform  simple  visual  and  psychomotor 
tasks  in  VEs.  We  then  examined  VE 
effectiveness  for  teaching  the  configura¬ 
tion  of  and  routes  through  large  build¬ 
ings,  and  the  transfer  of  the  knowledge 
acquired  to  the  real  world.  These  results 
led  to  basic  research  into  the  problems  of 
distance  estimation  in  VEs.  We  also  used 
VE  to  represent  exterior  terrain,  both  for 
training  land  navigation  skills,  and  trans¬ 
fer  to  the  actual  terrain. 

Recently,  we  investigated  the  value  of 
navigation  aids  for  improving  configura¬ 
tion  knowledge  acquisition  in  a  VE. 
Currently  we  are  examining  using  VE  for 
training  team  tasks  (see  Figure  1). 
Overall,  we  have  conducted  15  experi¬ 
ments  involving  690  participants.  Knerr 
et  al.  (1998)  summarize  this  research. 


In  each  experiment,  we  monitored  participants 
for  simulator  sickness  and  measured  their  symp¬ 
toms.  Simulator  sickness  refers  to  symptoms  result¬ 
ing  from  performing  a  task  in  a  simulator  that  do 
not  occur  when  performing  the  task  in  the  real 
world.  Symptoms  include  nausea,  dizziness, 
headache,  and  eyestrain  and  are  measured  by  a  16- 
item  Simulator  Sickness  Questionnaire  (SSQ) 
(Kennedy,  Lane,  Berbaum,  &  Lilienthal,  1993). 
Because  VE  exposure  produces  similar  symptoms, 
we  consider  VE  sickness  a  subcategory  of  simulator 
sickness.  VE  sickness  produces  discomfort  that  may 
distract  the  trainee,  possibly  interfering  with 
learning  and  training  transfer  and  reduc¬ 
ing  the  sense  of  presence  in  the  VE. 
Severe  discomfort  may  discourage 
further  user  participation  in  the 
training.  Aftereffects  involving  the 
I  sense  of  balance,  such  as  postural 
I  disequilibrium  (ataxia),  or  involv- 
f  ing  visual  flashbacks  could  possi¬ 
bly  impair  the  users'  ability  to  drive 
safely  or  perform  skilled  motor  tasks 
following  VE  exposure. 

— ^  VE  sickness  symptoms  have  been  suffi¬ 

ciently  severe  that  8.4%  (58/690)  of  our  partici¬ 
pants  have  withdrawn  from  experiments.  Attrition 
rates  for  individual  experiments  varied  from  0%  to 
25%.  The  experiments  with  high  attrition  rates 
tended  to  involve  frequent  self-motion  and  periods 
of  constant  VE  exposure  (i.e.,  exceeding  10  minutes 
between  breaks).  Figure  2  shows  that  for  experi¬ 
ments  with  substantial  attrition  rates,  SSQ  scores 
were  much  higher  for  participants  who  withdrew 
before  completing  the  experiments.  Some  who 
withdrew  exhibited  symptoms  within  the  first  five 
minutes,  although  average  time  before  withdrawing 
was  typically  longer.  For  example,  the  average  VE 
exposure  time  before  withdrawal  was  28  minutes 
for  participants  exposed  to  a  VE  over  several  ses¬ 
sions  interspersed  with  short  breaks. 

For  those  who  persevere,  the  severity  of  symp¬ 
toms  seems  to  accumulate,  at  least  up  to  a  point, 
over  time  of  immersion.  When  symptoms  were 
measured  before,  midway  through,  after  an  average 
immersion  of  30  minutes,  and  after  a  30-minute 
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recovery  period,  the  midpoint  SSQ  scores  were  sig¬ 
nificantly  higher  than  the  pre-immersion  scores,  but 
the  midpoint  and  post-immersion  scores  did  not  dif¬ 
fer.  During  the  30-minute  recovery  period  scores 
returned  to  pre-immersion  levels.  Another  study 
produced  a  sUghtly  different  pattern  over  five  VE 
immersions,  with  symptoms  increasing  significantly 
during  the  first  immersion  and  falling  back  during 
the  next  immersion.  VE  sickness  gradually 
increased  over  the  remaining  immersions  to  peak 
on  the  last  immersion. 

Predicting  Individiial  Susceptibility 

In  some  experiments,  participants  reported 
whether  they  had  ever  experienced  motion  sick¬ 
ness  and  rated  their  susceptibility.  Correlations  of 
these  items  with  post-immersion  SSQ  scores  were 
significant,  but  small  (.18  and  .16  respectively]. 
Although  it  appears  that  motion  sickness  suscepti¬ 
bility  is  related  to  VE  sickness,  VE  sickness  often 
occurs  in  the  absence  of  any  significant  motion. 
Recently  we  developed  a  14-item  screening  ques¬ 
tionnaire  targeted  at  predicting  VE  sickness.  While 
the  validity  of  this  questionnaire  has  not  been 
established,  preliminary  results  are  encouraging, 
with  significant  correlations  between  the  screening 
questionnaire  and  SSQ  scores  of  .38  to  .56. 

Practices  Adopted 

In  conducting  our  research,  we  have  adopted 
practices  to  reduce  simulator  sickness  among  our 
participants.  While  these  have  not  been  validated 
formally,  they  are  based  on  pilot  testing  and  our 
own  personal  experience  and  reactions. 

For  the  first  VE  exposure,  we  use  short  sessions 
(10-15  minutes)  interspersed  with  breaks  (5-10 
minutes).  If  the  participant  is  standing,  we  provide 
a  stable  surface  for  hand  contact.  Participants  are 
instructed  to  avoid  quick,  jerky  head  and  body 
movements.  The  room  is  kept  cooler  than  normal, 
with  fans  providing  air  movement  to  decrease  dis¬ 
comfort. 


Conclusions 

VE  sickness  can  negatively  affect  the  use  of  VE 
for  training.  Roughly  10%  of  the  population  may 
not  be  able  to  use  today’s  immersive  VEs.  Initial 
exposures  to  VE  should  be  short  with  frequent 
breaks,  and  recovery  periods  should  be  provided 
for  those  who  report  moderate  to  severe  symp¬ 
toms.  While  the  reduction  or  elimination  of  VE 
sickness  is  a  long-term  goal,  in  the  short  term  we 
need  better  tools  for  identifying  those  participants 
who  are  most  susceptible  to  VE  sickness.® 
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Figure  2.  SSQ  scores  for  users  who  compieied  or  withdrew  from  experiments. 
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Potential  Rehabilitation 
Benefits  of  Virtual  Reality 


Cheryl  Trepagnier 


¥irtual  Reality  (VR)  now  plays  a 
mainstream  role  in  automotive 
design  and  pilot  training.  Gaming 
has  brought  VR  displays  into  arcades 
and  homes.  Phobias  are  being  treated 
through  virtual  exposure,  with  results 
comparable  to  those  of  standard  treat¬ 
ment.  Patients  engaged  in  VR  activities 
are  finding  relief  from  the  pain  of  having 
burn  wounds  dressed.  Physicians  prac¬ 
tice  virtual  laparoscopic  surgery  before 
moving  on  to  the  real  thing,  and  sur¬ 
geons  operate  guided  by  virtual  images 
of  internal  organs  projected  on  the 
patient’s  skin. 

The  National  Rehabilitation  Hospital 
(NRH)  Rehabilitation  Engineering 
Service,  with  Michael  Rosen,  Ph.D.  as  its 
director,  has  a  professional  staff  of  seven, 
and  a  support  staff  of  two,  and  is  recruit¬ 
ing  new  staff  at  both  levels.  There  is 
ongoing  collaboration  with  NRH  physi¬ 
cians  and  psychologists,  and  with  col¬ 
leagues  from  the  Biomedical  Engineering 
and  Psychology  Departments  at  the 
Catholic  University  of  America. 

Like  many  groups  around  the  country 
and  abroad,  investigators  at  NRH  are 
looking  into  the  potential  benefits  of  VR 
for  individuals  with  disabilities.  The  pur¬ 
pose  is  not  to  offer  virtual  experiences  as 
a  substitute  for  interaction  with  the  real 
environment,  but  to  provide  evaluation 
and  training  that  will  support  the  indi¬ 
vidual’s  efforts  to  fulfill  his  or  her  own 
goals  in  the  real  world. 

Early  work  at  NRH  produced  a 
“Virtual  Shopping  Cart,”  successfully 
in  an  empathic  experience  for  users; 
the  goal  is  to  approximate  the  chal¬ 
lenges  confronting  the  person  with  dis¬ 
ability  (see  Figure  1).  In  this  case  the 
experience  of  visuospatial  neglect  is 
simulated  for  family  members  of  the 
individual  recovering  from  a  stroke. 
Visuospatial  neglect  can  be  thought  of 


as  a  failure  to  attend  to  stimuli  appearing  on  the 
side  opposite  to  the  affected  brain  hemisphere, 
particularly  when  the  stroke  has  damaged  the 
right  hemisphere.  In  practical  terms,  the  patient 
may  not  notice  someone  standing  on  her  left,  or 
may  fail  to  eat  the  food  on  the  left  side  of  his 
plate.  The  virtual  experience  has  two  modes  of 
operation.  In  “non-disabled”  mode,  the  VR  partic¬ 
ipant  steers  the  cart  through  shoppers  and  traffic 
while  fully  aware  of  the  surroundings.  Under  the 
visual  neglect  condition,  the  participant  fails  to 
see  what  is  on  the  left  side,  and  steering  becomes 
much  less  successful. 

A  current  focus  of  VR  work  at  NRH  involves  not 
only  display,  but  also  sensing  of  direction  of  gaze. 
An  eye-tracking  camera  has  been  installed  in  the  VR 
headset  so  that  the  direction  of  gaze  of  the  right  eye 
is  sampled  up  to  60  times  per  second.  This  system 
has  been  used  to  examine  stroke  patients’  attention 
to  objects  on  the  left  side  of  space.  A  study  now  in 
progress  uses  the  system  to  investigate  face  recog¬ 
nition  by  persons  with  autism.  Participants  are 
asked  to  look  at  three-dimensional,  static  images  of 
faces  and  objects.  They  are  then  shown  more 
images,  and  their  task  is  to  report  whether  they 
have  previously  seen  each  image,  or  not.  The  pri¬ 
mary  hypotheses  are  that  individuals  with  autism 
will  show  greater  impairment  in  face-recognition 
(relative  to  object-recognition,  as  compared  with 
controls)  and  that  their  gaze  patterns  will  differ 
from  controls’.  While  only  a  few  individuals  have 
participated  so  far,  preliminary  evidence  is  consis¬ 
tent  with  these  predictions.  Typically,  individuals’ 
face  recognition  accuracy  declines  when  the  faces 
are  presented  upside  down,  since  the  configural 
information  important  to  face  recognition  is  lost 
when  faces  are  inverted.  Unlike  controls,  persons 
with  autistic  disorder  are  no  worse  at  dealing  with 
inverted  faces  than  they  are  with  faces  in  their 
usual,  upright  orientation.  The  implication  is  that 
they  base  their  judgments  on  featural  rather  than 
configurational  information. 

Findings  of  anomalous  gaze  suggest  that  train¬ 
ing  in  effective  face-gaze  may  be  a  useful  compo¬ 
nent  of  intervention  for  persons  with  autism.  If 
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Figure  1.  Operating  the  Virtual  Shopping  Cart  to  gain 
empathy  with  stroke  patients. 


dier  deposited  in  alien  territory  is  an 
appropriate  one  for  the  individual  with 
autism  trying  to  cope  with  the  social 
world.  The  behavior  of  others  is  as  inex¬ 
plicable  and  unpredictable  for  persons 
with  autism  as  the  actions  of  members  of 
an  unfamiliar  culture  are  to  the  young 
peacekeepers. 

The  long-term  prospect,  we  hope,  will 
be  social  skills  experiences  provided  over 
the  internet  to  individuals  with  autism, 
in  their  own  homes,  at  times  they  prefer. 
Eventually  it  may  be  possible  for  indi¬ 
viduals  with  autism  to  preview  new  situ¬ 
ations,  practice  appropriate  behavior  for 
attending  a  concert,  buying  groceries,  or 
rehearsing  job  interviews,  so  that  they 
can  take  on  the  outside  world  with 
reduced  anxiety  and  with  a  greater  like¬ 
lihood  of  connecting  with  others,  bs 


For  additional  information, 
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the  data  continue  to  bear  out  our  hypotheses,  that 
is  an  avenue  we  will  pursue. 

Training  in  social  skills  is  also  of  vital  importance 
for  individuals  with  autism.  For  many,  social  skills 
deficits  represent  the  major  barrier  to  employment. 
VR  offers  an  exciting  way  to  approach  teaching  and 
practicing  these  skills.  Situations  can  be  presented 
that  could  be  dangerous  if  encountered  in  real  life. 
One  goal  of  training  must  be  to  reduce  the  likeli¬ 
hood  of  the  sort  of  behavior  that  leads  to  victimiza¬ 
tion,  or  causes  others  to  become  fearful,  call  the 
police,  or  even  resort  to  violence.  A  large  number 
and  variety  of  scenarios  can  be  presented,  and 
rewards  can  be  built  in,  so  that  the  individual  is 
motivated  to  engage  in  the  therapy  often  and  at 
length.  A  therapist  need  be  involved  only  in  an 
executive  role— reviewing  data  downloaded  from 
the  system  to  monitor  progress  and  interest,  and 
selecting  new  scenarios  when  warranted. 

A  project  just  beginning  at  NRH  is  the  develop¬ 
ment  of  the  “Virtual  Mall.”  Using  software  devel¬ 
oped  for  the  training  of  emergency  and  military  per¬ 
sonnel  who  encounter  culturally  unfamiliar  and 
threatening  situations,  the  Virtual  Mall  will  be  pop¬ 
ulated  with  virtual  human  beings  who  will  respond 
to  the  participant's  direction  and  speed  of  move¬ 
ment,  contact,  and  speech.  The  metaphor  of  the  sol- 
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®  EYE  MOVEMENT  MONITORING 
SYSTEMS 


ISCAN  is  the  world's  leading  producer  of  video  based  eye 
movement  monitoring  systems.  Since  1980,  our  broad  range 
of  products  has  set  the  pace  for  performance  and  usability  in 
wide  ranging  research,  military  and  medical  applications. 
We  can  provide  off-the-shelf  or  customized  solutions  to  suit 
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Non-invasive,  high  resolution,  real¬ 
time  measurement  of  eye  position,  pu¬ 
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Complete  Eye  and  Head  Tracking 
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Specialized  Remote  Mounted  Eye  Imaging  Systems  for  Long  or 
Short  Range  Applications. 

Advanced  PCI  Bus  Eye  Tracking  Electronics. 

Complete  Data  Acquisition  Hardware  and  Analysis  Software  for 
Determination  of  Subject's  Visual  Performance. 
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